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Abstract: We have tested and deployed Artificial Neural Network (ANN) data mining 
techniques to analyze remotely sensed multi-channel imaging data from MODIS, GOES, 
and AVHRR. The goal is to train the ANN to learn the signatures of wildfires in 
remotely sensed data in order to automate the detection process. We train the ANN using 
the set of human-detected wildfires in the U.S., which are provided by the Hazard 
Mapping System (HMS) wildfire detection group at NOAA/NESDIS. The ANN is 
trained to mimic the behavior of fire detection algorithms and the subjective decision- 
making by NOAA HMS Fire Analysts. We use a local extremum search in order to 
isolate fire pixels-, and then we extract a 7x7 pixel array around that location in 3 spectral 
channels. The corresponding 147 pixel values are used to populate a 147-dimensional 
input vector that is fed into the ANN. The ANN accuracy is tested and overfitting is 
avoided by using a subset of the training data that is set aside as a test data set. We have 
achieved an automated fire detection accuracy of 80-92%, depending on a variety of 
ANN parameters and for different instrument channels among the 3 satellites. We 
believe that this system can be deployed worldwide or for any region to detect wildfires 
automatically in satellite imagery of those regions. These detections can ultimately be 
used to provide thermal inputs to climate models. 

1. Introduction 

Wildfires have a profound impact upon the biosphere and our society in general. They 
cause loss of life, destruction of personal property and natural resources and alter the 
chemistry of the atmosphere. In response to the concern over the consequences of 
wildland fire and to support the fire management community, the National Oceanic and 
Atmospheric Administration (NOAA), National Environmental Satellite, Data and 
Information Service (NESDIS) located in Camp Springs, Maryland gradually developed 
an operational system to routinely monitor wildland fire by satellite observations. The 
Hazard Mapping System, as it is known today, allows a team of trained fire analysts to 
examine and integrate, on a daily basis, remote sensing data from Geostationary 
Operational Environmental Satellite (GOES), Advanced Very High Resolution 
Radiometer (AVHRR) and Moderate Resolution Imaging Spectroradiometer (MODIS) 
satellite sensors and generate a 24 hour fire product for the conterminous United States. 
Although assisted by automated fire detection algorithms, NOAA has not been able to 
eliminate the human element from their fire detection procedures. As a consequence, the 
manually intensive effort has prevented NOAA from transitioning to a global fire product 
as urged particularly by climate modelers. NASA at Goddard Space Flight Center in 
Greenbelt, Maryland is assisting NOAA more fully automate the Hazard Mapping 
System by training neural networks to mimic the decision-making process of the fire 



analyst team as well as the automated algorithms. 


2. Data Archiving 


Two years ago, the Computing, Information and Communications Technology (CICT), 
Program operating out of the Ames Research Center in 
Moffett Field, California, provided funding for the research effort to get underway. A 
team of government and (ultimately) University personnel were assembled with the intent 
of applying artificial intelligence techniques to NOAA’s automation problem. NASA 
began archiving satellite imagery from GOES, AVHRR and MODIS satellite sensors in 
the summer of 2003. Three spectral channels for each of 3 science instruments were 
provided by NOAA NESDIS by uploading to a NASA computer within the Information 
Systems Division at Goddard Space Flight Center. The spectral bands, being only a 
subset of those available from each instrument were found to be the most useful in fire 
identification by NESDIS and are shown in Table 1. Both reflectance and brightness 
temperature were scaled by NESDIS to a range of 0 - 255. 


Table 1. Spectral Bands Provided by NESDIS 


GOES 

AVHRR 

MODIS 

0.62 pm (chi) 
3.9 pm (ch2) 
10.7 pm (ch4) 

0.66 pm (chi) 
0.91 pm (ch2) 
3.7 pm (ch3b) 

0.66 pm (chi) 
0.86 pm (ch2) 
3.96 pm (ch22) 


It became apparent almost immediately that the huge volume of satellite imagery which 
NESDIS provided but did not archive themselves presented a formidable storage 
problem. Each day, NESDIS processes 96 GOES images files, 25 AVHRR image files 
and 16 MODIS image files which were uploaded to NASA. Satellite imagery, in Lambert 
Conformal Conic Projections, plus ancillary data required approximately 1.44 gigabytes 
of storage daily. A Mac G5 with 12 Maxtor external (250GB) disk drives was able to 
handle the enormous storage requirement. 

2. Preliminary Analysis 

The preliminary analysis consisted of a series of scatter plots starting with GOES 
imagery to determine separability of fire and non-fire clusters. An example is shown in 
Figure 1 in which clusters of background pixels (upper) and fire pixels (lower) are 
distinguishable in a GOES, channel 1 (reflectance) and channel 2 (brightness 
temperature) scatter plot for a particular pixel over the course of a single day. Intensities 




are background subtracted. This was the first indication, confirmed by subsequent 
analysis, that different fires types (crown, surface and ground) did not have unique fire 
signatures and that a simple linear separability existed between the two classes of fires 
and non-fires. 


Figure 1 Scatter Plot of GOES Channels 1 and 2 



CHI (background subtracted) 


3. Data Reduction 

A good deal of thought, time and attention went into the composition of adequate neural 
network training sets. Early attempts included time and geographic location parameters in 
addition to spectral information but due to subsequent difficulties in obtaining 
convergence, ultimately only the spectral information was used. The original guiding 
principle in training set composition was to use NOAA’s ASCII data- formatted fire 
product to locate fires within satellite imagery, then extract 3 -band pixel information at 
these points. 

Table 2 depicts the appearance of NOAA’s fire product in ASCII data format. As shown, 
on each line and for each separate fire, the geographic coordinates of the fire are followed 
by a time stamp, the satellite imagery from which the determination was made and finally 



the method of detection which may have been a human analyst or one of the automated 
algorithms: Wildfire Automated Biomass Burning Algorithm (WF-ABBA) [1,2], Fire 
Identification Mapping and Monitoring Algorithm (FIMMA) [1,2], MODIS MOD 14 [3] 
Fire product (MODIS). 


Table 2. ASCII Data Format of NOAA Fire Product (as of 05/16/03) 


Lon 

Lat 

Time 

Satellite 

Method of Detect 

-80.597 

22.932 

1830 

MODIS AQUA 

MODIS 

-79.648 

34.913 

1829 

MODIS 

ANALYSIS 

-81.048 

33.195 

1829 

MODIS 

ANALYSIS 

-83.037 

36.219 

1829 

MODIS 

ANALYSIS 

-83.037 

36.219 

1829 

MODIS 

ANALYSIS 

- 85.767 

49.517 

1805 

AVHRR NOAA- 16 

FIMMA 

-84.465 

48.926 

2130 

GOES-WEST 

ABBA 

-84.481 

48.888 

2230 

GOES-WEST 

ABBA 

-84.521 

48.864 

2030 

GOES-WEST 

ABBA 

-84.557 

48.891 

1835 

MODIS AQUA 

MODIS 

-84.561 

48.881 

1655 

MODIS TERRA 

MODIS 

-84.561 

48.881 

1835 

MODIS AQUA 

MODIS 

-89.433 

36.827 

1700 

MODIS TERRA 

MODIS 

-89.750 

36.198 

1845 

GOES 

ANALYSIS 


Using a software package called Environment for Visualizing Images (ENVI), 
geographic coordinates were converted to pixel row and column coordinates for a 
particular image being processed through a series of ENVI function calls embedded in 
IDL code. When examining fires using ENVI in visual mode however it was found that 
fires were not in the precise location where the geographic coordinates placed them, 
being offset possibly by several pixels from their expected location. Considering the 1 
kilometer spatial resolution of MODIS and AVHRR, the offset error might have been 2 
or 3 kilometers but for GOES data in the thermal band (4 KM resolution) the error could 
have been as much as 12 kilometers. This offset was attributed to 3 sources: spacecraft 
navigation errors, the inherent tolerances within NOAA software and operational errors 
in the point-and-click method of a Fire Analyst identifying fire locations with a mouse. 

One of the best clues for identifying wildfires that NOAA Fire Analysts employ is to 
visually inspect the 4 micron band for dark spots within NESDIS-processed satellite 
imagery. NOAA software has been written in such a way -that brightness temperatures, 
which have been scaled to a range of 0 - 255 will assume the lowest values for the 
hottest fires. This can be seen clearly in Figure 2a. which displays a GOES Channel 2 
satellite image of Northern Florida, Julian day 126 in 2003. A fire at coordinates -82.10 
degrees West Longitude, 30.49 degrees North Latitude is shown (enclosed in rectangle). 
Although appearing to be a pinpoint location in the normal view, a zoom-in in Figure 2b. 
indicates that the fire is actually spread across numerous pixel locations. In order to 
extract the spectral information around this exemplar fire, using the approximate location 




as specified by the ASCII data fire product, our software performed a local minima 
search in the 4 micron band in the expected region to pinpoint the hottest fire pixel 
(lowest intensity value). Spectral information was then collected around that image 
coordinate in all 3 bands. 


Figure 2a. GOES CH 2 
Northern Florida Fire 


Figure 2b. GOES CH 2 (Zoom) 
Northern Florida Fire 



Three different methods to characterize a fire across 3 spectral bands were investigated: 
as a single pixel at an instantaneous point of time, a pixel time series demonstrating the 
time evolution of a fire throughout the day and as a pixel array at an instantaneous point 
in time. The first two techniques had mixed results in achieving neural network 
convergence, however the third, a spatial technique consisting of 7x7 pixel arrays with 
the hottest part of the fire as the central pixel, was successful. In the 4 micron band, the 
approximate location of hot spots were identified by the NOAA’s fire product, then the 
local minima technique identified the hottest part of the fire. In all 3 bands, using the 
image coordinates of the hottest fire pixel, pixel arrays of size 7x7 were collected around 
that central point. A typical spatial fire pattern for the MODIS sensor is shown in Table 3. 
Numeric values represent reflectance or brightness temperature scaled to a 0 - 255 range. 
In the 3.96 pm band the pattern becomes visually obvious. Moving away from the central 
* pixel, the cooler parts of the fire are represented by rising intensity values (an inversion 
by design). 



4. Neural Network Architecture 

Three bands of 7x7 pixel arrays, formatted as 147 element vectors determined the 
number of network input nodes while the number of hidden nodes was initially 
determined by the rule-of-thumb to start with the square root of the sum of the inputs 



Table 3. Typical 3 Channel MODIS 7x7 Pixel Array Spatial Fire Signature 

CHI (0.66 pm) 


70 

65 

65 

73 

74 

71 

66 

81 

76 

80 

68 

67 

61 

63 

74 

75 

74 

75 

75 

61 

62 

63 

71 

80 

81 

79 

66 

63 

62 

69 

77 

78 

77 

69 

59 

69 

75 

69 

78 

77 

67 

72 

85 

82 

65 

69 

67 

72 

79 

CH2 (0.86 pm) 

139 

156 

155 

125 

133 

135 

145 

151 

143 

141 

129 

129 

137 

142 

146 

143 

143 

136 

129 

145 

142 

144 

146 

128 

127 

128 

138 

142 

148 

144 

138 

124 

125 

134 

145 

140 

145 

147 

123 

123 

138 

131 

129 

136 

148 

141 

144 

146 

136 


CH22 (3.96 pm) 


46 

51 

48 

35 

35 

38 

48 

41 

38 

35 

41 

43 

51 

50 

46 

41 

34 

20 

42 

53 

52 

52 

21 

3 

0 

21 

51 

51 

51 

36 

4 

28 

43 

49 

56 

41 

42 

50 

48 

41 

49 

42 

28 

35 

47 

47 

49 

43 

37 . 


and outputs, i.e.12. Even with some experimentation though, the number of hidden nodes 
did not vary much from the initial value. A single output node was required to 
discriminate between the 2 classes. The 147-10-1 supervised, feedforward 
backpropagation neural network configuration used for training and testing is shown in 
Figure 3. Separate, identical networks were created for each sensor. Hyperbolic tangent 
transfer functions were selected for all active nodes. 





Defining equations of the network for levels 0, 1 and 2 can be simply expressed by the 
following notation: 
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1 
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O = f( I ) 

1 1 


where: 

I = Induced Local Field 

0 = Nodal Output 

I, O superscripts = layer number 
I, 0 subscripts = node index number 
W = connectionist weight 
W superscripts = destination layer 

W subscripts = destination and source node index numbers, respectively 
N = total number of nodes per layer 

1 = input node index number (1 to 147) 
h = number of hidden nodes (10) 

j = hidden node index number (1 to 10) 
f = tanh(x) 


Figure 3, 147-10-1 MODIS, GOES or AVHRR FFBP NEURAL NETWORK 
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5. Neural Network Training and Testing 


Thousands of spatial pattern training samples were extracted from the 3 sensor imagery 
which temporally spanned the 2003 fire season across the continental United States. 
Three different neural network modeling tools were used in the course of the 
investigation: Java Object Oriented Neural Engine (JOONE), Stuttgart Neural Network 
Simulator (SNNS) and MATLAB Neural Network Toolbox however the following 
discussion pertains only to results obtained with the MATLAB tool. 

For each of the satellite instruments/sensors, the total available samples of spatial patterns 
extracted from satellite imagery is shown in Table 3, the ratio of fires to nonfires being 
approximately 1:1. A variation of the cross-validation technique [4] was employed for 
training and testing. Total available patterns for each instrument were divided into 4 
quarters, each being representative of the entire data set. Training samples constituted Vi 
of the total number of patterns with l A relegated to a validation set and Va a test set 
resulting in 3 disjoint data sets. Batch training using the Gradient Descent with 
Momentum algorithm was selected from a suite of the available MATLAB routines and 
to prevent overfitting, early stopping was employed (though not in all cases). During 
training, the mean squared error on the validation set was monitored and when it began to 
rise, training was automatically halted. Testing then continued on data that had not been 
seen by the neural network during the training phase. 


Table 3 Total Available Samples 


MODIS 

AVHRR 

GOES-EAST 

GOES-WEST 

25,713 

43,758 

73,010 

53,922 


5. Classification Results 

Results of the neural network classification for MODIS, AVHRR and GOES data are 
presented below in the form of error matrices in Tables 4-7 (see Congalton’s excellent 
discussion [5]). Analysis of the error matrices is discussed in the next section. Since this 
is a 2-class system, empirical data represents true positives (TP), true negatives (TN), 
false positives (FP) and false negatives (FN) which occupy the upper left hand comers of 
the error matrices. Remaining numeric data shown in the matrices are marginal totals. 

Table 4 MODIS Error Matrix 

Reference Data 
Fire NonFire 


Fire 

03 

03 

Q 

C NonFire 


O 


2834 

(TP) 

173 

(FP) 

3007 

318 

(FN) 

3103 

(TN) 

342 

1 

315 

3276 

6428 










Table 5 AVHRR Error Matrix 


Reference Data 


53 

03 

Q 

<d 

s 

CO 

53 

a 



Fire 

NonFire 


Fire 

550 

0 

47 

9 

597 

A 

NonFire 

62 

4 

433 

6 

496 

A 


6124 

481 

109 



C 

7A 


Table 6 GOES-WEST Error Matrix 


Reference Data 


Fire NonFire 



Table 7 GOES- EAST Error Matrix 

Reference Data 


Fire NonFire 






5. 


Statistical Analysis 


The error matrices of Tables 4-7 were analyzed statistically in the manner of Congalton 
et al. [5] with results tabulated in Table 8. In terms of true positives (TP), true negatives 
(TN), false positives (FP) and false negatives (FN), 5 measures of accuracy were 
calculated as follows: 


Overall Accuracy 
Producer’s Accuracy (fire) 
Producer’s Accuracy (nonfire) 
User’s Accuracy (fire) 

User’s Acuracy (nonfire) 


(TP+TN)/(TP+TN+FP+FN) 

TP/(TP+FN) 

TN/(FP+TN) 

TP/(TP/FP) 

TN/(TN+FN) 


This was followed by a Kappa analysis [5] in which the Khat statistic determines the 
degree of agreement between classified data and reference data. The Overall Accuracy 
measure reflected the neural networks ability to assign unknown image pixels to either of 
two classes, fire and background while the Producer’s and User’s Accuracy figures 
provided accuracy measures for individual classes. As Table 8 indicates, a high degree of 
classification accuracy (-90% Overall and 87% - 94% Producer’s/User’s) was achieved 
for the MODIS and AVHRR sensors while classification of GOES image data (mid 70% 
- mid 80%) performed relatively poorly. This was attributed to the accuracy of the 
science instruments themselves as well as the refinement in fire detection algorithms 
which followed the earlier GOES methods. KHAT values which have a range of +1 to -1 
indicated positive correlations in all cases while MODIS and AVHRR classifications 
achieved a strong agreement (greater than 80%) and GOES (low to mid 60% range) 
again lagged behind. 


Table 8 Statistical Analysis of Neural Network Classification 


STATISTIC 

MODIS 

AVHRR 

GOES 

WEST 

GOES 

EAST 

Overall Accuracy 

92.3615 

89.9168 

83.5163 

80.1885 

Producers Accuracy (fire) 

89.9112 

89.8106 

76.8839 

85.0177 

Producers Accuracy (nonfire) 

94.7192 

90.0519 

89.2961 

75.7346 

Users Accuracy (fire) 

94.2468 

91.9886 

86.2248 

76.3668 

Users Accuracy (nonfire) 

90.7045 

87.4194 

81.5933 

84.5702 

Khat 

0.847041 

0.796063 

0.666441 

0.60486 









6 . 


Conclusions 


The original intent of the project was to develop a single neural network that could 
process sensor data from all 3 instruments, MODIS, AVHRR and MODIS, perform fire 
classification at least as well as the automated algorithms and human fire analysts 
currently achieve and be incorporated into NOAA’s operational Hazard Mapping System 
to reduce the amount of manual intervention. Our research has shown that there was 
insufficient temporal and spatial overlap between the 3 sensors to process image data 
with a single network nor would it have been practical at any rate due to the extraordinary 
network size which was exacerbated by the 7x7 pixel arrays to characterize fire patterns. 
The practical solution was to divide processing between 3 independent networks however 
the low classification accuracy for GOES EAST and GOES WEST imagery suggests that 
only MODIS and AVHRR imagery need be processed by the neural network. In spite of 
the high classification accuracy that was achieved for MODIS and AVHRR sensor data, 
further improvement is likely to be achieved by incorporating additional generalization 
techniques that MATLAB offers: Modified Performance Function, and Bayesian 
regularization. Further research effort is still required before the neural designs could be 
incorporated into an operation al system. 
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